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(57) ABSTRACT 

Methods, apparatus, and systems for optimized digital photo 
management include an optimized digital photo manage- 
ment user interface, referred to as the photo-journal UI 
(PJUI), a client application with a photo-journal UI for 



general purpose personal computers (of any form factor), 
referred to as a photo-journal application, a dedicated por- 
table client device with a small-screen optimized photo- 
joumal UI, referred to as a photo-tablet, and a companion 
server for either photo-joumal client platform, referred to as 
a tablet-server. Thus, the photo-tablet and the photo-journal 
application are alternative photo-journal client embodiments 
with different variations of the same photo-journal user 
interface architecture. The photo-tablet is designed to be an 
intermittently connected Internet-appliance. In the photo- 
tablet, the small-screen optimized PJUI runs entirely on the 
photo-tablet, even for actions involving the server. Thus the 
photo-tablet docs not require a separate personal computer, 
web-browser, or other Internet access means. The photo- 
joumal application client isolates the lay user from com- 
puter-centric technical details. The photo-journal application 
client also may be configured for intermittent (deferred) 
access. User requests for network services (e.g., printing) are 
queued in either photo -journal client until the next connec- 
tion. When a photo-journal client is connected to the Inter- 
net, it finds a tablet-server, identifies itself, uploads any new 
photos, and uploads any orders for prints or other services. 
Photo-journal client software is automatically updated by 
the server to track changes in features and options. Photos 
that have been added to the tablet-server from other sources, 
such as a film processing and scanning provider or a PC, are 
automatically downloaded to the photo-journal client. 
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APPARATUS, METHODS, AND SYSTEMS FOR 
DIGITAL PHOTO MANAGEMENT 

FIELD OF INVENTION 

[0001] This invention relates to management of collec- 
tions of digital photos and to associated user interfaces. 

BACKGROUND 

[0002] It is expected that the huge consumer photography 
market will make a massive technology transition during the 
next decade. Driven by fundamental technology trends, 
digital cameras will steadily and rapidly take over the 
majority of the camera market. Just how fast this transition 
will occur is a subject of some debate, but there is little doubt 
that the transition will take place. It is estimated that 4. 7 
million digital cameras were sold worldwide in 1999, and 
that this figure will increase to 22 million in 2003. 

[0003] Today, using a digital camera is impractical if you 
are not a capable computer user. This is because digital 
camera users are presently required to manage their digital 
images using the hierarchical-directory file-centric paradigm 
that is native to present personal computers. Some printers 
can connect directly to a camera or read flash memory cards 
to access pictures, but computer skill is still required to store 
and manipulate the pictures. Photo-sharing Web sites, which 
have proliferated in great numbers, provide valuable ser- 
vices for digital camera users but are only part of a complex 
solution currently suitable only for the computer sophisti- 
cate. Photo-printing sites again provide only part of the 
solution. As digital cameras become mainstream, the limi- 
tations of file-centric management of digital photos could 
become crippling. 

[0004] As digital cameras spread beyond photo enthusi- 
asts to snapshot takers, there will be many potential camera 
customers with no desire (or ability) to deal with computer- 
specific technical details. What is needed is a digital photo 
solution with appliance -like simplicity, which snapshot tak- 
ers will be capable of using and want to deal with, and which 
can access the Internet to perform comprehensive end-to- 
end digital photo management. 

[0005] Previous Computer Based Solutions. 

[0006] Digital photo -management functions are done 
today by personal computers. To do so, however, requires an 
assortment of programs and peripherals. The complexity of 
the task includes learning each of the programs, managing 
hierarchical directories of photo files, dealing with a printer, 
finding printing services, and ensuring reliable backups. It is 
far too cumbersome, daunting, and hard to learn for typical 
photo consumers (not enthusiasts) to adopt in significant 
numbers. File-centric personal -computer-based photo man- 
agement no doubt will improve greatly in the coming years, 
but the file-centric paradigm is very unlikely to evolve 
rapidly into one that most snapshot consumers will find 
satisfying. Digital cameras will become pervasive, but they 
need a much better companion photo management solution 
than the file-centric approach to achieve this goal. What is 
needed is a comprehensive end-to-end unified digital photo 
management solution for typical photo consumers. 

[0007] Personal computers have significant capabilities 
for working with digital photos. They typically have large 
screens, which provide for dramatic viewing and allow 



many pictures to be seen at the same time. Photoediting 
programs on personal computers offer extensive capabilities, 
although very few people have the skills to use such pro- 
grams or want to invest the amount of time required in each 
picture. Today, the person who edits photos on their personal 
computer is more akin to a home darkroom enthusiast than 
to a typical photographer. What is needed is a means for a 
typical photographer to enjoy the fruits of extensive photo- 
editing capabilities with minimal effort, training, or invest- 
ment. 

[0008] Personal computers also provide significant flex- 
ibility to create photo albums, Web pages, and other publi- 
cations. Today's software requires that such creations either 
conform to very simple templates or that the user have a 
high-end design program and the talent to use it. For the 
most part, however, the more sophisticated capabilities are 
beyond what most photo users are interested in. What is 
needed is a means to enable most photo users to publish their 
photos in a wide range of electronic and print formats with 
minimal effort, training, or investment, 

[0009] A notebook computer is a portable platform that 
can be used to share photos. Because of its physical clum- 
siness, however, taking out a notebook at a gathering to pass 
around is quite awkward. Hie keyboard gets in the way, and 
it is not easy for users to browse through a "stack" of photos. 
What is needed is a portable platform for sharing photos that 
is easy to pass around and permits users to easily browse a 
stack of photos. 

[0010] Pen-based computers designed for field workers, 
such as those made by Fujitsu, Casio, and Mitsubishi, are 
candidates for use in digital photo management. However, 
such devices have all the same ease-of-use barriers inherent 
in conventional personal computers — in addition to being 
too expensive. The industrial tablet computers are expensive 
(typically $2,000 and up) in part because they use large LCD 
displays. These devices also lack interfaces that are desirable 
for digital photo users, such as SmartCard and Memory 
Stick interfaces, and, in some cases, audio input/output and 
TV/monitor output interfaces. What is needed is a computer 
solution with appliance-like simplicity for digital photo 
management that inexpensively provide easy, convenient, 
and robust operation. What is further needed is a computer 
solution for digital photo management that includes inter- 
faces for removable memory, audio, and TV (or other video). 

[0011] Photo Sharing and Printing Web Sites 

[0012] There has been a proliferation of photo-sharing and 
printing Web sites, such as ofoto, zing, gatherround, shut- 
terfly, activeshare, photopoint, photoloft, photoisland, club- 
photo, and iprint. Most of these sites will store arbitrary 
numbers of photos for the user. Most broker print services, 
while others are primarily front ends for internal printing 
operations. 

[0013] Unfortunately, these Web sites are only part of a 
complex solution currently suitable only for the computer 
sophisticate. Photos must be manually uploaded to the site; 
they have no concept of maintaining a synchronized archive. 
The sites are built around a focus of sharing selected photos, 
not of managing all your photos for your own use. Interact- 
ing with them is slow and tedious unless the user has a fast 
Internet connection. Even with a fast connection, loading a 
significant number of pictures is time consuming, and the 
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browser interface does not provide an easy or reliable 
method for doing this in the background. What is needed is 
an Internet-based service for digital photo management that 
provides an easy and reliable means of managing synchro- 
nized photo archives for individuals, including the capability 
for automatic background photo transfers (uploads and 
downloads). (Such a digital photo management service 
would likely also offer photo sharing and photo printing 
services. ) 

[0014] Camera-Printer Systems 

[0015] Several makers of photo printers, recognizing the 
difficulties of current computer-reliant solutions, support 
direct printing of photos without using a computer. In some 
cases, the camera is connected to the printer via a serial 
interface, or sends the image data via IR (infrared); in others, 
the memory card is removed from the camera and inserted 
into the printer. The printer can automatically generate an 
index page and then produce prints in the selected sizes and 
quantities. 

[0016] This approach has two major limitations. First, it 
provides no way to store the pictures. Flash memory cards 
are far too expensive to use as a long-term storage medium, 
and in any case keeping the pictures organized would be 
quite problematic. Second, this approach is limited to local 
printing. Although prints from low-cost photo printers have 
become very good, they are still not quite the same as a 
silver halide print, and they are time-consuming and expen- 
sive to produce. Many users will prefer to use Internet-based 
printing services, which will deliver priDts in a day or two 
using conventional photo paper with very little hassle. What 
is needed is a long-term storage solution for digital photos 
that additionally facilitates printing either via a local printer 
or via a remote printing service. 

[0017] Camera -Internet Interfaces 

[0018] There are products in development that will enable 
cameras to communicate directly with photo-sharing Web 
sites, allowing photos to be uploaded without use of a 
personal computer. FotoNation, which was acquired by 
Zing, has announced plans to sell such a product under the 
name FotoCall. These products could be useful for sending 
images while traveling, but since they provide no photo 
display without using a computer (or other Web access 
device), it does not provide a complete solution. 

[0019] Zing, FotoNation, and Casio have developed per- 
sonal computer software that will automatically upload 
photos from a camera connected via USB and transfer these 
photos to the Zing site without user intervention and without 
putting the photos on the computer's hard drive. This 
solution provides no mechanism for locally managing your 
photos. 

[0020] Some camera makers are expected to include 
modems in some future cameras, enabling the camera to 
communicate directly with a photo-sharing site. The lack of 
any local storage for the photos, however, combined with the 
time-consuming transfers that keep the camera busy, are 
likely to make this an unappealing solution for most users. 
What is needed is an appliance-like solution for digital photo 
management and associated services that: provides for pro- 
grammed photo transfers to and from remote storage without 
requiring direct user supervision, does not reduce camera 
availability, and does not require the user to be a computer 
sophisticate. 



[0021] Photo Management Systems 

[0022] There are commercial photo-management systems, 
such as Digitella's IntelliPix, that maintain photo archives 
and will synchronize the archive with other devices (typi- 
cally personal computers). Such systems use off-the-shelf 
hardware and are designed for large commercial users, such 
as real estate agents. The genesis of theses systems in the 
commercial environment will make it difficult for them to 
reach the cost and ease-of-use points needed for the con- 
sumer market. What is needed is a consumer-oriented digital 
photo management system supporting synchronized digital 
photo archives. 

[0023] Portable Photo Viewer Devices 

[0024] A portable photo viewer product has been sold by 
Video Chip Technologies, but it simply displays pictures 
from a memory card and provides no storage or printing 

[0025] functions. Priced at $399, VideoChip's "Wallet" 
viewer with a 4" diagonal display became available in June 
2000. The company plans other products with larger dis- 
plays but has no announced plans for products with mass 
storage or photo-organizing or enhancement functions, nor 
is there any Internet connection. Panasonic, Samsung, and 
others have showed concept prototypes of Flash-memory- 
based viewers. 

[0026] Sony offers an $899 digital photo frame (called the 
Cyberframe) that provides some digital photo management 
fictions in a desktop device. As with the portable photo 
viewer, however, the Cyberframe provides no solution for 
long-term storage, organization, or use of Internet services. 
It also supports only Sony's Memory Stick storage cards. 
What is needed is a portable photo viewer that incorporates 
functionality for display, long-term storage, printing, photo- 
organizing, photo enhancement, and access to associated 
Internet services. 

BRIEF DESCRIPTION OF DRAWINGS 

[0027] FIG. 1 is a system diagram showing a photo- 
journal client (such as a photo-tablet or a photo-journal 
application running on a personal computer) and photo 
management server (a tablet-server), interconnected via the 
Internet, and in accordance with the present invention. 

[0028] FIG. 2 is a block diagram of a photo display and 
management tablet (a photo-tablet) in accordance with the 
present invention. 

[0029] FIG. 3 is a system block diagram illustrating the 
use of a conventional TV for displaying photos managed on 
a photo-journal client platform. 

SUMMARY 

[0030] The present invention teaches the use of an opti- 
mized digital photo management user interface, referred to 
as the photo-journal UI (PJUI). The PJUI is a client user 
interface intended for use with a companion server, referred 
to as a tablet-server. The PJUI is implemented in two 
illustrative and alternative client embodiments, a photo- 
journal application, and a photo-tablet. The photo -journal 
application runs on general-purpose personal computers as a 
client application implementing the PJUI. Depending on the 
form factor and available human interface I/O, the PJUI of 
the photo-journal application may be further optimized as 



10/21/2003, EAST Version: 1.04.0000 



US 2002/0087546 Al 



3 



Jul. 4, 2002 



appropriate. The photo-tablet is a dedicated portable client 
device with a small-screen optimized PJUI. 

[0031] The photo-tablet is designed to be an intermittently 
connected Internet-appliance. In the photo-tablet, the small- 
screen optimized PJUI runs entirely on the photo-tablet, 
even for actions involving the server. Thus the photo-tablet 
does not require a separate personal computer, web-browser, 
or other Internet access means. The photo-joumal applica- 
tion client isolates the lay user from computer-centric tech- 
nical details. The photo-journal application client also may 
be configured for intermittent (deferred) access. 

[0032] User requests for network services (e.g., printing) 
are queued in either photo-journal client until the next 
connection. When a photo-journal client is connected to the 
Internet, it finds a tablet-server, identifies itself, uploads any 
new photos, and uploads any orders for prints or other 
services. Photo-journal client software is automatically 
updated by the server to track changes in features and 
options. Photos that have been added to the tablet-server 
from other sources, such as a film processing and scanning 
provider or uploaded over the Internet from personal com- 
puters, are automatically downloaded to the photo-journal 
client. If a photo archive is maintained on the tablet-server, 
then the photo-journal client preferably maintains a photo 
cache subset. However, market research as determined that 
many users are uncomfortable with either a partial or 
complete remote archive, so the user's photo archive may 
instead be archived on the photo-journal client, either exclu- 
sively, or redundantly with the tablet-server. The foregoing 
system features are illustrated in FIG. 1. 

[0033] This instant application focuses primarily on the 
photo-tablet embodiment, which was initially detailed in U. 
S. Provisional Patent Application Ser. No. 60/179,379, 
entitled "DIGITAL PHOTO MANAGEMENT INTERNET 
APPLIANCES AND ASSOCIATED METHODS AND 
SYSTEM," Michael Slater et. al., filed Jan. 31, 2000, also as 
previously incorporated by reference. The PJUI architecture 
generally, and its use as in the photo-journal application 
embodiment, are detailed as provided in U.S. Provisional 
Patent Application, entitled "PHOTO JOURNAL USER 
INTERFACE," Ken Rothmuller et. al., filed Jan. 16, 2001, 
as previously incorporated by reference. Differences may 
exist between the applications with respect to various user 
interface features. Unless indicated to the contrary, the Jan. 
31, 2000 application and this instant application may have 
features more specifically optimized to a small-screen por- 
table platform, while the Jan. 16, 2001 application is gen- 
erally more platform-generic but reflects a greater experi- 
ence with the user interface design. Those skilled in the art 
will be able to pick and choose features from the two 
applications based on this knowledge and in view of their 
particular implementation constraints. 

[0034] For small-screen portable platforms, a fully dedi- 
cated hardware implementation of the photo journal user 
interface, such as provided by the photo-tablet, is a preferred 
approach for those applications in which performance and 
portability are both very important. While the costs are 
expected to decrease rapidly over the next few years, the 
present cost of the dedicated hardware implementation is 
expected to limit the overall market for such a device. A fully 
software implementation of the photo journal user interface, 
such as provided by a photo journal application running on 



a desktop personal computer, is a preferred implementation 
if a larger market reduced cost solution is desired. 

[0035] Another small-screen portable platform solution is 
to implement the photo journal application on pen-based 
handheld-platform that have sufficient resolution to permit 
enjoyable casual viewing of color digital photos. (An 
example of such a platform is the Compaq iPaq running the 
Microsoft PocketPC OS). This solution provides the benefits 
of the photo-tablet form factor and does not incur the cost of 
the filly dedicated hardware embodiment. This solution is a 
compromise in that computer-specific technical details are 
only hidden from view only while the photo -journal appli- 
cation is running. Such platforms are also presently quite 
expensive by themselves, such that this market is also quite 
limited in size. 

[0036] In the context of the foregoing, the instant patent 
application makes many comparative references between 
the photo-tablet and photo management solutions based on 
the PC. Unless explicitly indicated otherwise, for economy 
of presentation, references to PCs and PC-based solutions 
refer to prior personal computer approaches (of any OS and 
machine vendor type) that are not based on the photo-journal 
user interface and are not found in the handheld form factor. 
Those skilled in the art will be able to easily appreciate 
which of these comments are not applicable to a photo- 
journal application running on a desktop or handheld PC. 

[0037] A photo -tablet, in accordance with the present 
invention, is intended to satisfy the diverse digital photo 
management needs of typical next-generation digital photo 
consumers. The needs satisfied by the present invention 
include: 

[0038] a system enabling users to share their photos 
with others that requires no technical computer skill 
and is easy for lay users; 

[0039] a system enabling users to share their photos 
with others that decouples the user's requests to 
perform sharing from the underlying communica- 
tions processes that actually perform the sharing; 

[0040] a system enabling users to share their photos 
with others that neither requires immediate execu- 
tion of the underlying communications processes, 
nor requires attended operation during the commu- 
nications processes; 

[0041] a portable photo viewer for sharing photos 
that is easy to pass around and permits users to easily 
browse a (virtual) stack of photos; 

[0042] programmed automatic background photo 
transfers with optional remotely networked long- 
term storage for maintaining synchronized photo 
archives for individuals; 

[0043] inexpensive equipment and services that have 
easy, convenient, and robust user operation, but do 
not require direct user supervision for task execution, 
do not reduce effective camera availability to the 
user, and do not require the use of a PC; 

[0044] local interfaces for removable memory, and, 
in some cases, audio, and TV (or other video); 

[0045] printing via either a local printer or a remote 
printing service; 
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[0046] enjoyment of extensive photo-organizing and 
photo enhancement (photoediting) capabilities with 
minimal effort, training, or investment; and 

[0047] publishing of user photos in a wide range of 
electronic and print formats and to a wide range of 
destinations with minimal effort, training, or invest- 
ment. 

[0048] The apparatus, systems, and methods for digital 
photo management in accordance with the present invention, 
include, but are not limited to: Internet appliances, Internet 
servers, networked collections of multiple appliances and 
servers, Internet-enabled services, transaction protocols, 
associated subsystems, associated program methods and 
data structures, and associated signals-in-flight and media. 

[0049] The photo-tablet provides a complete, streamlined, 
easy-to-use, and highly effective solution to digital photo 
management It is a photography information appliance that 
stores photos, displays them, and works in concert with an 
Internet server — accessed automatically by the photo-tab- 
let — for backup, extended storage, and printing. 

[0050] The photo-tablet serves as a portable photo album 
with a (virtualized) vast store of photos. It is a comprehen- 
sive solution to the needs of digital camera users, eliminat- 
ing the complexity of a PC. In addition to all the other 
benefits of digital photography, it solves a widespread prob- 
lem that plagues both digital and film camera users: how to 
organize and share your photos — not just over the Web, but 
also in person, wherever you are. It also provides all the 
benefits of Web-based photo sharing sites and printing 
services in a faster and easier way than using a PC. 

[0051] The photo-tablet has fundamental advantages over 
a PC-based digital photo management approach. It has 
exactly the interfaces and software needed, in the smallest 
practical physical configuration, with integrated Web sup- 
port. Not only does a photo-tablet do most of what a user 
would do with a PC, with a superior user interface; it also 
provides a portable means of displaying photos and offers a 
complete solution for digital photo management. 

[0052] With a digital camera and a photo-tablet, or a 
hybrid photo-tablet/camera, consumers can easily store and 
organize their pictures. It provides secure long-term storage 
with virtually no effort. The photo-tablet need only be 
occasionally connected to the Internet, and it will automati- 
cally archive new photos to the tablet-server and process 
requests for prints. No browser is needed; interactions with 
the server are handled automatically by the photo-tablet. The 
user interface runs online on the photo-tablet, making it very 
responsive and hiding all the delays of Internet access. 

[0053] The photo-tablet enables users to create simple, 
elegant albums, and special companion Web sites will 
provide automatic creation of Web pages. The photo-tablet 
enables users to have access to all the capabilities of these 
sites without the need for a PC. Because the user interface 
runs off-line and the Internet connection is designed to run 
intermittently without user intervention, it will be more 
pleasing to use. The user interacts with the photo-tablet, the 
photo-tablet interacts with the server in the background. 
And, of course, the photo-tablet provides a portable device 
for viewing photos anywhere. 

[0054] A photo-tablet may be as small as a paperback 
book or as large as a magazine, depending on the display 



size. FIG. 2 shows an illustrative embodiment that includes 
an LCD display with touchscreen, slots for various types of 
memory cards used in digital cameras, and a hard drive to 
store photos. The illustrated embodiment also provides a 
USB interface for an optional local printer, or for direct 
connection to a camera. The embodiment shown also 
includes NTSC (TV) and RGB (monitor) outputs to allow 
photos to be viewed on a larger screen. The photo-tablet also 
preferably includes some manner of network connection, 
such as the illustrated modem port for a dial-up Internet 
connection, or an Ethernet or Bluetooth link to a network 
gateway that couples to a LAN, intranet, or the Internet. 
FIG. 3 illustrates how the photo-tablet of FIG. 2 may 
display photos on a TV without a direct cable coupling. 
Here, the photo-tablet is directly coupled to an interface that 
couples via a data link (such as a wireless or powerline data 
link) to a set-top box coupled to the TV. 

[0055] Many different devices can be built with different 
cost/size/feature tradeoffs. A first illustrative embodiment 
includes a roughly 4 ,, x6" display, mimicking the size of a 
common print while minimizing size and cost. A second 
embodiment, designed to be used mostly at one location but 
to roam freely within a modest distance, is a device with a 
10-14" display anda wireless connection to a base station. 
The base station translates the wireless link to an Internet 
connection and serves as a charging stand for the photo- 
tablet. Initial photo-tablets will use such small displays, 
because of their reduced cost. Embodiments of the photo - 
tablet designed primarily as desktop (or wall- mounted) 
viewing devices are also in accordance with the present 
invention. 

[0056] The photo-tablet provides everything a consumer 
needs to easily store and organize their pictures, share them 
with friends in person or via the Web, and get prints either 
from their own printer or a service bureau. The user needs 
only to provide a phone line or other Internet connection 
periodically, and the photo-tablet will automatically archive 
any new photos to the tablet-server and process any requests 
for prints or other services. No browser is needed; the user 
interface runs offline on the photo-tablet. 

[0057] The photo-tablet makes it easy for a user to share 
photos, in person as well as via the Web, without prints. 
Today, this can be done at a PC, but there is no convenient 
way to take pictures to a friend's house, or even to the living 
room. 

[0058] The primary benefits of the photo-tablet include: 

[0059] Easy management of all the user's photo- 
graphs 

[0060] Sharing of photos in person, at any location 

[0061] Automated off-site backups for secure long- 
term storage 

[0062] Easy access printing services and Web sharing 
sites 

[0063] A photo-tablet can work with a PC, for users who 
want to use PC-based photo manipulation programs. The 
photo -tablet can connect directly to the PC and, through a 
small utility running on the PC, maintain a synchronized 
image database on the PC. All the benefits of the photo- 
tablet, including automatic off-site archiving, easy access to 
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print and sharing services, and portable display of photos are 
available to the PC user as well. 

[0064] Because digital cameras are still more expensive 
than film cameras, and because the process of getting prints 
is much more complex, digital photography is currendy 
limited to an enthusiast market. Only early adopters are 
willing to put up with the complexity and clumsiness of 
managing their photographs using a PC. The photo-tablet 
provides the first truly easy to use, complete solution for 
digital photography. 

[0065] Digital camera users today are, by definition, com- 
puter literate enough to manage their photos on a PC. But 
many of them will prefer the simple, portable, reliable, and 
effective solution offered by the photo-tablet. Even if they 
continue to use a PC for photo editing or generating publi- 
cations, the photo-tablet is valuable as a portable viewing 
device, and photo-tablet software running on the PC will 
ensure automatic back-up to the Web site. The photo-tablet 
can also be used to collect photos in the field, enabling an 
arbitrary number of pictures to he stored by periodically 
loading the pictures from the camera to the photo-tablet. It 
makes a great companion to a PC for users who are already 
adept at using a PC to work with photos. 

[0066] Another advantage of the photo-tablet is its supe- 
rior user interface and integration. PC photo -management 
products typically are clumsy and overburdened with fea- 
tures, and often several products must be used to perform the 
tasks performed by the photo-tablet. The photo-tablet 
approach provides easy categorizing of photos and auto- 
mated assembly of virtual photo albums that is superior to 
any PC software product available today. 

DETAILED DESCRIPTION 
[0067] Photo-tablet Operation 

[0068] A illustrative photo -tablet usage scenario is as 
follows: 

[0069] Photos are removed from digital camera on a 
memory card 

[0070] The memory card is inserted in the photo- 
tablet 

[0071] The photo-tablet automatically loads the pic- 
tures on to its hard disk 

[0072] User browses the photos, adjusts or deletes 
photos as needed, assigns keywords, and optionally 
records voice annotations 

[0073] The next time the photo-tablet has Internet 
access, the new photos are automatically uploaded to 
the tablet-server 

[0074] In an illustrative embodiment, the photo-tablet 
software allows the user to enter keywords in four catego- 
ries: people, things, events, and places. Each photo is also 
tagged with a date stamp. The photo-tablet comes with a 
common set of pre-programmed keywords for each cat- 
egory, and users can enter their own using a touch-screen 
keyboard. Once a photo -tablet has been customized with 
names of the user's family and common travel destinations 
or photo- taking locations, most keyword assignments 
require only touching the appropriate words from an on- 



screen list. Users can also identify favorite photographs (the 
best from each event, trip, etc. ). 

[0075] Photos can be arranged automatically into albums, 
based on keywords, making it easy to create a "slide show" 
with a particular set of photographs. Printed albums can be 
produced on a local printer or can be produced by a 
tablet-server companion site arid mailed. 

[0076] By recording a voice annotation for selected pho- 
tos, a story can be associated with each image. Thus, the oral 
history that corresponds to each photo, all too often lost once 
a picture is a few years old, is captured forever. 

[0077] In a preferred embodiment, the photo-tablet will 
optionally perform automatic image enhancement, adjusting 
color saturation, brightness, and contrast. With a simple user 
assist (identifying the eyes), red eye can be automatically 
eliminated. 

[0078] The photo-tablet is designed to be an intermittently 
connected Internet device. In a preferred embodiment, the 
user interface runs entirely on the photo-tablet, even for 
actions involving the server. User requests for network 
services (typically printing) are queued in the photo-tablet 
until the next connection (and the user is warned if much 
time has gone by with no connection, ) When the photo- 
tablet is connected to the Internet, it finds the photo-tablet 
server, identifies itself, uploads any new photos, and uploads 
any orders for prints or other services. Photo-tablet software 
is automatically updated by the server to track changes in 
features and options. Photos that have been added to the 
tablet-server from other sources, such as a film processing 
and scanning provider or a PC, are automatically down- 
loaded to the photo-tablet. 

[0079] The photo-tablet is typically set to connect to the 
tablet-server late at night; it need only be connected to the 
tablet-server (e.g., via phone line dial-up to an Internet 
Service Provider) sometime before the scheduled hour. 
Thus, it requires Internet access only when people are 
generally sleeping, making relatively long connect times 
acceptable even without a dedicated line. As Table 1 shows, 
using the best-quality JPEG images from a 2-Mpixel camera 
yields an upload time of about 45 minutes for 10 pictures, 
and almost 3 hours for 36 pictures. With even a slow DSL 
or cable connection (384 Kbits/s), 36 pictures will take only 
12. 5 minutes, and 100 pictures (a busy day) would take only 
35 minutes. With a 1. 5-Mbit/s connection, which should be 
common by 2003, even 100 pictures will take less than 10 
minutes. 

TABLE 1 



Upload times, in minutes, for various connection speeds, assuming 
high-quality JPEG images from a 2-Mpixel camera and 80% utilization of 
the peak channel speed. 









Modem 


DSL/Cable 


Fast DSL 


Number of 


Picture 




28.8 


384 


1.5 


Pictures 


Size 


Total Size 


Kbits/s 


Kbits/s 


Mbits/s 


10 


800 K 


8 M 


46 


3.5 


0.9 


36 


800 K 


28.8 M 


167 


12.5 


3.2 


70 


800 K 


56 M 


324 


24.3 


6.2 


100 


800 K 


80M 


463 


34.7 


8.9 



[0080] Increasing camera resolution will increase these 
times, but resolutions much beyond 3 Mpixels are unlikely 
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to be popular in the next several years. Wavelet-based 
compression algorithms (planned for JPEG 2000) will pro- 
duce much greater compression ratios, significantly reduc- 
ing upload times. 

[0081] The pictures are instantly available on the photo- 
tablet, without any need for a Wet) connection and without 
communication delays, since they are stored on the photo- 
tablet's hard disk. At the same time, they are automatically 
archived off-site, providing protection against loss and 
enabling access (password protected) to the photos from any 
Internet connection. 

[0082] The tablet-server provides the primary archive for 
all photos. Any individual photo-tablet may not have enough 
storage to contain all the user's photos; if there are more 
photos on the server than can fit in the photo-tablet, the 
photo-tablet's disk is treated as a cache. Typically, it would 
hold all the most recent photos, though the user can choose 
other selections (such as flagging certain groups of photos to 
always be included). Even a modest hard disk drive can hold 
thousands of high-resolution digital images, and it will not 
be long before economical hard disks will store tens of 
thousands of images. 

[0083] Io its highest-quality JPEG mode, a 2-megapixel 
camera produces images of approximately 800 Kbytes. A 
small hard disk today is 6 Gbytes, equivalent to 7,500 such 
images. With JPEG 2000 compression, the number of 
images may be increased more than tenfold. 

[0084] The disk capacity can be substantially increased, at 
the user's option, if standalone local printing is not required. 
For this type of user, high-resolution image files are archived 
on the tablet-server, with only a screen-resolution image 
stored in the photo-tablet. Local printing can still be per- 
formed by downloading the required high-resolution files 
from the tablet-server. 

[0085] The photo-tablet can also store photos delivered to 
the Web site from any source, enabling it to handle film- 
based photos as well as digital photos. The tablet-server can 
communicate with film-processing sites that deliver scanned 
images from film, photo-tablets can communicate with each 
other, or with a PC, through wired (USB), RF (Bluetooth), 
or IR connections. With a single command, the photos stored 
in two or more devices can be synchronized. Photo libraries 
(or individual photos or albums) can also be loaded into any 
photo-tablet from the tablet-server via the Internet. 

[0086] Using photo-tablet PC software, photo-tablet own- 
ers who also have a PC can maintain a synchronized copy of 
their photo library on their PC's disk. Photos can be edited 
on the PC, and the modified copies automatically saved to 
the tablet-server (and then mirrored to the photo-tablet the 
next time it connects). 

[0087] In a preferred embodiment, photos or photo albums 
can be printed to a local printer, connected via the photo- 
tablet's USB port. 

[0088] The photo-tablet can optionally connect to a tele- 
vision or monitor for large-screen display. This option will 
become particularly attractive with the deployment of digital 
TV receivers, with their higher-quality displays. 

[0089] Internet access for the photo-tablet will be provide 
by an ISP with an existing network of dial-up connections. 
As cable modems and DSL connections become widespread, 



along with home networks, the photo-tablet will be able to 
optionally connect via the home network. 

[0090] Photo-tablet Hardware 

[0091] Photo-tablets can be built in many different hard- 
ware configurations. Preferred embodiments share the fol- 
lowing basic features and capabilities: 

[0092] Display: Backlit LCD with touch screen. 

[0093] User Interface: Touch screen plus a few hard 
buttons 

[0094] Memory Card Interfaces: SmartCard, Com- 
pact Flash, and Memory Stick 

[0095] Peripheral and PC Interface; USB port (and 
possibly a 1394 port) 

[0096] Mass Storage: hard drive, flash memory, or 
other technology 

[0097] Modem (or other form of Internet connection) 

[0098] Optional TV and RGB outputs 

[0099] Microphone and speaker 

[0100] Optional IR or RF (Bluetooth or 2.4 GHz) 
interfaces 

[0101] Photo-tablet Implementation 

[0102] In a particular illustrative embodiment, the photo- 
tablet has chassis features similar to a color Palm-size PC 
such as Casio's Cassiopeia, a Windows CE handheld com- 
puter. Such color Palm-size PCs sell today for $500 and have 
adequate color displays, touch screens, standard modems, 
and processing power for a photo -tablet. However, the 
photo -tablet will be approximately twice as thick as present 
color Pam-size PCs to accommodate a hard disk, a larger 
battery, and additional memory card (e.g., SmartCard and 
Memory Stick) interfaces. 

[0103] FIG. 2 shows a block diagram of an embodiment 
of a photo-tablet in accordance with the present invention. 
The central block, which includes the processor, system 
logic, and I/O controllers, will ultimately be built as a single 
chip. Early implementations will require three to six chips 
for these functions, in order to use standard processor and 
system logic chips and programmable logic, rather than 
ASICs. 

[0104] In another embodiment, the photo-tablet has a 
complete single-board design created to provide exactly the 
features needed without additional assemblies. This embodi- 
ment will not require any custom ICs. 

[0105] In a preferred embodiment, the photo-tablet incor- 
porates nearly all the electronics on a single custom chip. In 
a another preferred embodiment, the photo-tablet uses a 
standard processor and a custom companion chip. This latter 
embodiment will reduce cost, size, and power consumption. 

[0106] In a preferred embodiment, the photo-tablet hard- 
ware will use PC-compatible processors and system logic, 
leveraging the wide range of available products, but with not 
intention of maintaining full PC compatibility. An x86 
microprocessor, while not the fastest, least expensive, or 
lowest power choice, will provide the best access to soft- 
ware. In addition, integrated processors with most of the 
required peripherals will be available. Even today, Nation- 



10/21/2003, EAST Version: 1.04.0000 



US 2002/0087546 Al 



7 



Jul. 4, 2002 



al*s Geode chip provides most of the electronics required for 
the photo-tablet in a single device. Within two years, there 
will be a variety of such chips available. The photo-tablet 
will operate quite adequately with processor performance 
one or two generations behind the PC mainstream. 

[0107] Photo-tablet and Tablet-Server Software 

[0108] The photo-tablet is not just a device but is an 
integrated solution combining a device and an Internet 
server. Thus, there are two software components, one for the 
photo-tablet and one for the server. 

[0109] Id a preferred embodiment, the photo-tablet user 
interface software is developed using conventional high- 
level language tools. The photo-tablet user interface binaries 
should be run on a real-time operating system, such as Wind 
River's VxWorks, which will provide the basic Internet 
connectivity code as well as a user interface toolkit and file 
manager. Libraries need to be licensed (or developed) for 
photo manipulation functions, such as JPEG compression 
and decompression, resampling, brightness and contrast 
adjustment, and other image enhancements. (An Adobe 
spin-off, PicturelQ, is currently licensing such software for 
embedded devices. ) The core software will be stored in 
Flash memory, enabling the system to run regardless of the 
state of the hard disk. Updates to the system software will be 
automatically loaded from the tablet-server when they are 
available and can be stored either on the disk or in the Flash 
memory. 

[0110] All state information will be saved to the tablet- 
server each time it connects to a photo-tablet. Even in the 
event of a catastrophic disk failure, the photo-tablet can be 
fully restored by the tablet-server, except of course for any 
photos that have not yet been uploaded, as long as the 
hardware is still functional. In the event of a hardware 
failure, a user can restore all their images to a new photo- 
tablet. 

[0111] Tablet-server 

[0112] The tablet -server provides services to the photo- 
tablet. It is similar to a Web site but it communicates directly 
with the photo -tablet without requiring a browser or any user 
intervention. In a preferred embodiment, functions per- 
formed by the tablet-server include: 

[0113] Photo-tablet Management: Check software 
versions of photo-tablets when they connect and 
update software as needed. 

[0114] Picture Storage: Perform automatic backup of 
photos added to photo -tablet since last connection 
(and send to the photo- tablet any new photos added 
to the tablet-server from other sources, such as a PC). 

[0115] Picture Printing: Print in various sizes and 
qualities, on specialty items as well as on paper, 
using an assortment of vendors. The tablet-server 
will automatically translate print requests to the 
requirements of the appropriate vendor and commu- 
nicate the order electronically. Formatted albums 
may be printed as well. (In most cases, these services 
will be provided by special companion Web sites. ) 

[0116] The special companion Web sites provide access to 
the user's photos from any computer, through a conventional 
Web interface. This site also allows photos to be shared with 
others via the Web. 



Conclusion 

[0117] Although the present invention has been described 
using particular illustrative embodiments, it will be under- 
stood that many variations in construction, arrangement and 
use are possible consistent with the teachings and within the 
scope of the invention. For example, bit-widths, clock 
speeds, and the type of technology used may generally be 
varied in each component block of the invention. Also, 
unless specifically stated to the contrary, the value ranges 
specified, the maximum and minimum values used, or other 
particular specifications, are merely those of the illustrative 
or preferred embodiments, can be expected to track 
improvements and changes in implementation technology, 
and should not be construed as limitations of the invention. 
Functionally equivalent techniques known to those skilled in 
the art may be employed instead of those illustrated to 
implement various components or sub -systems. It is also 
understood that many design functional aspects may be 
carried out in either hardware (i.e., generally dedicated 
circuitry) or software (i.e., via some manner of programmed 
controller or processor), as a function of implementation 
dependent design constraints and the technology trends of 
faster processing (which facilitates migration of functions 
previously in hardware into software) and higher integration 
density (which facilitates migration of functions previously 
in software into hardware), 

[0118] The invention is further not limited to a specific 
expansion module technology as used in the illustrative 
embodiments. In specific but not limiting examples, the 
invention is equally applicable to any of the present and 
future variants of the CompactFlash (including any of the 
Type I, Type II, and proposed Type III variants), PC Card 
(including any of the 32-bit, 16-bit, Type I, Type II, and Type 
III variants), and Springboard (or other open-back expansion 
module) standards, as well as other removable expansion 
module standards and technologies. 

[0119] The invention is further not limited to a specific 
removable memory (or media) technology as used in the 
illustrative embodiments. In specific but not limiting 
examples, the invention is equally applicable to the use of 
present and future variants of MMCs, Miniature Cards, 
Memory Sticks, SSFDCs, Smart Cards, and SIM Cards. 

[0120] The invention is further not limited to a specific 
network technology, topology, or scale. In specific but not 
limiting examples, the invention is equally applicable to 
networks at any scale including networks characterized as 
local area, departmental, enterprise wide, metropolitan area, 
state wide, regional, national, and the Internet. 

[0121] All such variations in design comprise insubstan- 
tial changes over the teachings conveyed by the illustrative 
embodiments. The names given to interconnect and logic are 
illustrative, and should not be construed as limiting the 
invention. It is also understood that the invention has broad 
applicability to other applications, and is not limited to the 
particular application or industry of the illustrated embodi- 
ments. The present invention is thus to be construed as 
including all possible modifications and variations encom- 
passed within the scope of the appended claims. 
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We claim: 

1. A method of operating a server to provide immediate 
and deferred response services to remote clients, the method 
including: 

a) providing dedicated server-side storage areas for client 
archives; 

b) implementing database management policies, said 
polices being configurable to each client archive; 

c) implementing fully automated communications with 
the client for data synchronization and client request 
submission; 

d) during the automated communications, dynamically 
executing downloads based on data synchronization 
needs, any outstanding deferred responses from 
requests received previously, and any immediately ser- 
viceable responses from current requests; and 

e) following the receipt of client uploads, selectively 
performing server-side processing based on the client 
archive specific database management policy and any 
client requests requiring deferred responses. 

2. The method of claim 1, wherein the services are 
explicitly customer-account based. 

3. The method of claim 1, wherein the database manage- 
ment policies include image management policies and the 
image management polices include methods of image syn- 
chronization. 

4. A method of operating a client device and an archive 
server, the method including: 

a) performing intermittent automated communications 
from the client to the archive server for image sync and 
request submission; and 

b) performing intervals of client operation without com- 
munications with the server, said client operation in 
intervals without communications including 

i. performing selective display of locally held images; 

ii. buffering images and requests received locally since 
previously having communications with the archive 
server, wherein said requests include requests to 
reconfigure image management policies of the 
server/client combination; and 

iii. indefinitely deferring communications of the buff- 
ered images and requests. 

5. A method for hiding from the user the process of 
archiving binary files to a remote server (including hiding 
the latencies of establishing a connection, account login, and 
slow modem transfer speeds, and including hiding the 
processes of enabling, supervising, and terminating the 
transfer), by quickly automatically transferring files from 
removable media, indefinitely holding the files in a buffer, 
and automatically transferring the data to a remote server in 
accordance with a predefined user profile, 

6. A method ("order and forget") of requesting services 
that unburdens the user from supervising the submission of 
the request for services (including hiding the latencies of 
establishing a connection, account login, and slow modem 
transfer speeds, and hiding the processes of enabling, super- 
vising, and terminating the transfer). 

7. A method of communicating images from a user- 
camera to a user-controlled archive using an intermediate 



storage device, wherein the user-camera-to-intermediate 
device operation is decoupled from and is generally asyn- 
chronous with the intermediate -device-to-archive operation, 
the latter occurring automatically after a variable delay 
following the former. 

8. The method of claim 7, wherein the intermediate 
storage device is local to the user and the archive is remotely 
accessed via a network. 

9. The method of claim 7, wherein the network is the 
internet. 

10. A method of automatic processing of remote services 
associated with digital photos, the method including: 

a) accepting and buffering a user request on a client 
platform; 

b) waiting a dynamically determined interval until a 
predetermined set of criteria are satisfied; and 

c) communication of the request and the associated digital 
photos from the client to a server, routing the request to 
a service provider, execution of the request, return 
routing of the result/response, and receipt of the result/ 
response by the client 

11. The method of claim 10, wherein the predetermined 
set of criteria includes that a request is pending and the 
current time is within a previously programmed time win- 
dow. 

12. The method of claim 1, further wherein the server acts 
in response to client requests as a single consolidated storage 
destination for multiple image sources other than the client, 
including: 

a) scanned images from new film developing; 

b) scanned images from conversion of existing prints; 

c) digital images provided by others via the internet; and 

d) image library services accessible via the internet. 

13. The method of claim 1, further wherein communica- 
tions to the client image archive are additionally available 
via real-time web-browser access. 

14. The method of claim 1, wherein the server supports 
simultaneous access by multiple customers to respective 
image archives, some customers using attended access via 
browsers and that require immediate-responses and other 
customers using unattended access via client devices that 
permit deferred-responses, 

15. A method of operating a server, the method compris- 
ing: 

a) reserving dedicated server-side image storage areas 
corresponding to each of a plurality of intermittently- 
connected client devices having client-side image stor- 
age areas, each dedicated server-side image storage 
area including at least one image storage area reserved 
for long-term image storage; 

b) maintaining client-associated data, including storage 
management policy data; 

c) performing automated user-unattended communica- 
tions with the client device that have no requirement for 
the downloading of real-time user-interface related data 
(e.g. HTML); 

d) during the user-unattended communications, uploading 
image data selectively provided by the client device; 
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e) during the user-unattended communications, uploading 
request data selectively provided by the client device; 

f) during the user-unattended communications, selectively 
downloading images to the client device as a function 
of the client-associated data and the selectively 
uploaded client request data; and 

g) subsequent to an instance of the user-unattended com- 
munications during which data is uploaded, selectively 
processing the uploaded data as a function of the 
client- associated data and the selectively uploaded cli- 
ent request data. 

16. The method of claim 15, wherein the clients are 
associated with customer accounts and client activities result 
in account billing. 

17. The method of claim 1, but further wherein the 
database management policy acts to: 

a) attempt to keep the client-side storage as a strict subset 
of the server-side (archive) storage; 

b) discard the oldest created images in the client-side 
storage, as required to accommodate new images; 

c) discard the least recently accessed images in the 
client-side storage, as required to accommodate new 
images; 

d) lock user selected images in the client-side storage; 

e) provide restoration of the contents of the client-side 
storage in the event of a loss; and 

f) maintain screen resolution copies in the client-side 
storage and printer resolution copies in the server-side 
storage. 

18. The method of claim 1, but further wherein at least 
some of the client requests for image processing services are 
executed directly by the server. 

19. The method of claim 1, but further wherein the server 
communicates at least one image in the archive to a specialty 
organization for request execution. 

20. The method of claim 1, but further wherein the 
execution of at least some of the requests requires the 
generation and delivery of physical materials. 



21. The method of claim 4, but further wherein the client 
device is completely integrated into a single portable hand- 
held device. 

22. The method of claim 4, but further wherein the client 
device has a wireless handheld portion and an associated 
base-station portion, wherein the display and user interface 
are in the handheld portion and the base-station has client- 
side image store and the circuitry to communicate with the 
server. 

23. The method of claim 4, but further wherein the general 
appearance of the user-interface is determined by page 
descriptions stored in non-volatile memory in the client that 
are modified as required by dynamic data conditions in the 
client and the most recent downloads from the server. 

24. The method of claim 4, but further wherein the client 
has a first non-volatile storage area dedicated to photos and 
a second non- volatile storage area dedicated to the user 
interface. 

25. The method of claim 24, wherein the photo storage 
area is maintained on revolving media, while the user 
interface is maintained in flash memory. 

26. The method of claim 24, wherein the first and second 
storage areas are maintained in flash memories having 
separate write controls. 

27. The method of claim 4, but further wherein the client 
device acts as a posted-write buffer to receive the locally 
received images and let the source of the images be put back 
to use before the images reach their ultimate destination. 

28. The method of claim 4, but further wherein the client 
device receives the locally received images via a removable 
memory. 

29. The method of claim 28, but further wherein the 
images are loaded automatically as soon as the removable 
memory is inserted. 

30. The method of claim 4, but further wherein the client 
device receives the locally received images via an I/O 
connector. 

31. The method of claim 30, but further wherein the 
images are loaded automatically as soon as the I/O connec- 
tion is sensed as being active. 
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